feat(loop): stall detection + bounded resume for hung in_flight issues (issue #98) - #146
Merged
Merged
Conversation
…s (issue #98) An in_flight issue (feat/issue-N-* branch, no PR yet) used to be refused forever if its driver hung or died mid-session, silently starving the issue. loop-census.sh now flags stalled=N age_min=M when an in_flight issue's newest events.jsonl activity (task field "N" or "issue-N") is older than budget.stall_minutes (default 30; zero events never counts as stalled). loop-tick.sh resumes a stalled/half-done in_flight candidate (action=resume issue=N branch=...) instead of refusing it, reusing loop-daemon.sh's classify_debris verbatim (issue #111) for the debris check, bounded to 2 attempts via a new .claude/state/loop-resume-attempts.json sibling file; the 3rd stall escalates to needs-human instead of retrying, mirroring the existing per-issue attempt-budget escalation. Every transition (stall detected, resume attempt N, escalated) is best-effort logged via log-event.sh. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dget Review rejected the first cut of issue #98: the resume verdict was gated on advance_ready appearing inside census's in_flight set, but loop-census.sh makes those mutually exclusive (advance_ready requires branch=none, in_flight requires branch!=none) -- so the resume path was dead code, never reachable in production. Rework the verdict decision to scan census's in_flight=/stalled= output directly, independent of advance_ready, once a fresh branchless advance candidate has had first claim on the tick. Also drop action=resume from the dispatch-bookkeeping case that fed loop-issue-attempts.json, matching the header comment's documented intent that resume attempts live only in the sibling loop-resume-attempts.json and never charge issue #95's advance/feedback spend ceiling. Reworks loop-tick.test.sh scenarios 15-20 to respect the real advance_ready/in_flight invariant, and adds precedence, lowest-numbered, escalated-skip, and budget-isolation coverage plus an end-to-end scenario driven by the REAL loop-census.sh (not a hand-fabricated fixture) proving the resume path is reachable through the actual integration. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
robercano
approved these changes
Jul 16, 2026
This was referenced Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the remaining scope of issue #98 (the recovery half of loop-health), after #111 (PR #118) landed the driver-side post-exit debris classifier. This PR adds the census-side stall detection and tick-side bounded resume machinery — reusing #111's
classify_debrisstate vocabulary (empty/publishable/half-done/resumable) verbatim, never diverging it.What changed
loop-census.sh) — newstalled=N age_min=Mtelemetry line per in_flight issue: afeat/issue-N-*branch exists, no open PR, and the newestevents.jsonlactivity for the task (matched as both"N"and"issue-N"forms) is older thanbudget.stall_minutes(default 30, adapter-overridable). Zero events for a task is conservatively NOT reported stalled. Additive output — existing census lines unchanged.loop-tick.sh) — a stalled orhalf-done-debris in_flight issue now yields anaction=resume issue=N branch=<name>verdict (scanning census'sin_flight=/stalled=lines directly, independent ofadvance_ready) instead of being silently refused. Precedence: ceiling > feedback > fresh advance > resume. Bounded to 2 resume attempts tracked in a sibling resume-state file (NOT Loop: spend ceilings — per-tick caps, daily budget with auto-halt, stop-after self-disarm #95'sloop-issue-attempts.jsondispatch budget); the 3rd stall escalates to aneeds-humanlabel and stops retrying. An escalated candidate is skipped in favor of the next non-escalated one rather than wedging the tick.events.jsonlvialog-event.sh(best-effort, never blocks the tick), so the loop-health cockpit panel (Cockpit: loop-health panel — last tick, cadence, verdict history, stall detection #85) can render them.Driver-side resume DISPATCH (reconnecting a fresh implementer to the stalled worktree) is a documented follow-up, out of scope here; an unrecognized
action=resumeverdict degrades safely toaction=nonedownstream today.Gates & review
GATES_FILE=.claude/self/gates.jsonbuild / lint / test all green (loop-tick.test.sh71 checks,loop-census.test.sh29 checks,loop-daemon.test.sh75 checks unaffected, plus fan-out smoke).correctness(opus) andtests(sonnet) lenses — both APPROVE (consensus=all). Both sabotage-checked the new reachability + budget-isolation tests as non-vacuous.Closes #98.
🤖 Generated with Claude Code